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PREFACE 


The work described in this report was performed by the Space Sciences 
Division of the Jet Propulsion Laboratory. 
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ABSTRACT 


A few new algorithms for region growing in pictures were developed. 
These algorithms are a step toward finding a satisfactory solution to the 
image segmentation problem, and in-depth understanding of the problems of 
nonsemantic image segmentation. The algorithms utilize a sequential 
decision approach for region boundaries detection. The sequential decisions 
are supported by a stochastic algorithm that maintains local statistics of the 
region near the boundaries as the region grows. A few illustrations of the 
algorithm's performance are included. 
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I. INTRODUCTION 


The information associated with each picture element (pixel) is the 
integral of the light energy reaching the corresponding sensor element. 
Typically, this light is reflected from or originated on a small area on the 
surface of a three-dimensional (3-D) body that is geometrically projected 
through the camera lens system on the corresponding sensor element. The 
total surface of a 3-Day body may be projected on a few pixels. Our objective 
is to group (cluster) all the pixels upon which the surface of one 3-D body is 
projected into a region. This region will be 2-D domain on the picture. 
Clearly, the extent of one 3-D body is context and task dependent (these define 
the desired resolution). Many algorithms that use world model (semantics) to 
find regions have been developed (Refs. 1, 2, 3, and 4). However, the 
emphasis here is on the development of general purpose problem independent 
segmentation algorithms that will be in a bag of tricks (feature extractors) of a 
higher level pattern recognition or semantic system like those in Refs. 5 and 
6, which will use them as feature extractors for the specific problem domain. 

This work is one in a long sequence of works on clustering and region 
growing. The region growing problem is basic for pictorial pattern recogni- 
tion and data reduction; a few of the typical applications are given in Refs. 2, 
3, 6, and 7. Few nonsemantic region growing algorithms are given in 
Refs. 3, 6, 8, 9, and 10. The algorithms presented below are new in that 
they integrate a sequential decision approach with stochastic maintenance of 
the local statistics on the boundaries of the growing region. Statistical likeli- 
hood ratio type tests were used before (Ref. 8), but never with sequential 
decision. Reference 8 gives a broader survey of related works on region 
growing. 

The implemented system described below is another phase in upgrading 
automatic segmentation in the way of speed, computer resources require- 
ments, and reliability. Unfortunately, well defined and usable quality 
measures for the performance of segmentation or clustering algorithms is 
lacking in the literature. For that reason, the algorithm's performances are 
demonstrated on few sample pictures. We hope that this illustration coupled 
with the theory will substantiate and justify our claims for improvement. 
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The mathematical -statistical justification of the algorithm is commingled 
with the description of the decision algorithms. That way we hope the theory 
presentation will be tied organically to the algorithmical implementation. 

*=* 

II. THE HARDWARE SYSTEM 

A diagram of the minicomputer -based image analysis system, on which 
the algorithms are implemented, is given in Fig. 1. RAPID, which is a 
Random Access Picture Digitizer display and memory system, is described 
in detail in Ref. 11. Basically, RAPID stores a digitized video image into a 
256 X 256 (8-bit) byte memory array internal to RAPID. 

Th • real time computer can randomly access any byte of the digitized 
image in about 5 ps by specifying an 8 -bit line address and an 8 -bit column 
address. The content of a byte is referred to in this publication as G(i, j) 
where 0 < i S 188 is a column number, 0 i j S 244 a line number, and 
0 ^ G(i, j) < 255 is the actual byte content (the digitized video level). 


scene 



RS-170 COMPOSITE 

COMPOSITE VIDEO 



Fig. 1. Minicomputer-based image analysis system, block diagram 
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The cameras used in the system (Fig. 2) are pure solid state (Charge 
Injection Device) cameras, G.E. TN-2000, which substantially meet their 
specified noise level of about 0. 5%. These earner? s are equipped with auto- 
matic iris lenses that maintain the average video signal on the whole image 
nearly constant. The sensing plate is composed of 188 X 244 sensing 
elements. The video signal is digitized so that RAPID's memory element 
value G(i, j) will be proportional to the light energy falling on sensing element 
(i, j) of the plate. 


The video output of RAPID contains current image of the content of its 
memory. The content of the memory can be refreshed continuously from the 
camera's video signal or remain in a static condition. In either case, the 
computer can randomly read or write a picture element in 5 ps, as was 
mentioned above. 


In addition to the image display capabilities provided by RAPID, a 
RAMTEK RX-100A is interfaced to the minicomputer, and it provides gray 
level and color display, and graphic capabilities. Eventually, we plan to 
replace the RAMTEK display unit with another RAPID -like unit. 





Fig. 2. Charge Injection Device camera on the robot vehicle 


JPL Technical Memorandum 33-774 



HI. INITIAL REGION GROWING TYPES 


Two approaches to initial image segmentation were implemented in our 
system. The first is called the "scanning" region grower; its control structure 
is described in Ref. 8. It scans the whole image from the top left corner line 
by line down to the bottom right corner and outlines all regions. The second 
type of region growing is called the "blooming" region grower. It starts from 
an arbitrarily specified point somewhere on the image and grows a region 
around the initial point (Ref. 12). An application of the sequential sampling 
mechanism (to be defined shortly) to the scanning region growing approach is 
given in Section IV, which is identical to Ref. 10. However, the bulk of this 
paper describes region growers of the blooming type. We found the reliability 
of the blooming region growers to be superior to that of the scanning region 
growers. That is due mainly to the fact that concentrating on one region 
allows us to keep more relevant information on the region being grown, and 
hence to achieve more reliable decisions. Also, the multiple applications of 
the blooming region grower result in a full segmentation, which also allows 
for overlapping regions (see Fig. 3), The need for nondeterministic segmen- 
tations (and possible overlapping regions) was suggested in Refs. 13 and 14, 
and we believe it will be usable by a semantic picture analysis system of the 
type Ascribed in Ref. 5, and for second-pass analysis (Ref. 8). 

A. THE BLOOMING REGION GROWERS 

The input for these kinds of region growers are: (1) the image matrix 
G(i, j), (2) the starting point (i Q , jg), (3) various decision confidence 
thresholds which define the resolution sensitivity. The output of the region 
grower will include at least a Boolean matrix IN(i, j). Here, IN(i, j) = TRUE 
means that the algorithm found point (i, j) to belong to the region grown 
around (ig, jg), and IN(i, j) s FALSE means that the algorithm found (i, j) to 
be outside that region. 

The output may also include the Boolean array OUT(i, j). OUT(i, j) = 
TRUE means that there were some tests that suggested that (1, j) should not 
be included in the region grown around (ig, jg). Because of the statistical 
nature of the decision-making process, there can be points for which 
OUT(i, j) is TRUE, but later they were added to the region; e.g. , IN(i, j) is 
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A PARTIAL DESCRIPTION OF REGION RECORDS AND 
MATRIX DATA STRUCTURE 


Fig. 3. A partial description of region records 
and sparse matrix data structure 
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alto TRUE, We anticipate that the OUT array will be used for error 
recovery in future systems. 

The raw output in the Boolean array IN(i, j) ia incorporated by a simple 
procedure into a sparse matrix form; this procedure also generates an 
appropriate region record. The region record includes: (1) information on 
the region growing parameters — e.g., starting point and thresholds, 

(2) statistics of the region sufficient to compute easily the mean and variance, 
and the first few momenta of the 2-D shape and the gray level structure, and 

(3) the enclosing rectangle (the maximum and the minimum of the 1 and j 
values over the region). 

The sparse matrix has for each horizontal line that intersects the 
region a record describing the segment(s) of the region on that line (see 
Fig. 3). The sparse matrix data structure allows integration of the output 
from few calls on the blooming region grower; hence it allows analysis of the 
whole image. Note that the segment may overlap; hence, the sparse matrix 
organization provides for overlapping regions. The sparse matrix represen- 
tation is more compact than the Boolean array structure, but is somewhat less 
efficient in core and access than the chain encoding representation we used in 
Ref. 8. But it provides for more flexibility in the regions' organization. The 
balance was in favor of sparse matrix representation for our purposes. 

B. THE BLOOMING REGION GROWERS* DRIVER 

The description of the algorithms is broken into two parts. First we 
describe the driver, and later we describe statistical data maintenance and 
decision. This is done because there are six combinations of decision 
procedures, all of them fitting the same driver. 

C. THE DRIVING ALGORITHM 

As was mentioned earlier, we assume that G(I, J) is available, and that 
(Iq, Jq), the starting point, is specified. 

1 , Initializatio n 

11 : Set IN(i, j) and OUT(i, j) to FALSE everywhere, 

12: IN(I 0 , J Q ) * TRUE 
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13: Gen< rate a new region record for a region containing only (Ig, Jq) 

14: Collect the statistic of the initial point (1 q, J^/. The statistic 

collected depends on the decision procedure and will be described 
later. 

15: Generate a, list of candidate point records. A record in that list 

specifies an image point still outside the region (its IN value is 
FALSE) that is adjacent to the region, and is a candidate to be 
added to the growing region. In the current phase, the candidate 
point records l'st contains at most 4 records for the points 
<1 0 - 1, J Q ), (I 0 + 1, J Q ) (I Q , J Q + 1) and (1 0 , J Q - 1). If any of 
these points is outside the digitized video addressing range, no 
candidate record is generated for that point. The format of the 
candidate point record is dependent on the decision algorithm and 
will be described later (it may contain some statistical informa- 
tion on the gray value of points in the region near the candidate 
point). 

2. The main loop 

Ll; If the candidate point records list is empty, the algorithm 
terminates. 

L2: Remove the candidate record on top of the candidate list. 

L3: Call the decision algorithm to test whether to add to the region 

the point specified in the candidate record just removed. If the 
decision is to add, continue in L5. 

L4: Let (i, j) be the point in the candidate record that was removed, 

then set OUT(i, j) s TRUE, and continue to Ll. 

L5 : Let (i, j) be the point in the candidate record that was 

removed, then set IN(i, j) = TRUE, and update the region record 
to include the new point. 

L6 : Test the four points (i - 1, j), (i + 1, j), (i, j - 1 ), (i, j + 1) (the four 

neighbors of (i, j), which was just added to the region), and decide 
whether to generate a candidate point record for any of those four 
points, and generate an appropriate record if one is suggested. 
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There are two decision options as to whether to generate a 
candidate record: (1) generate a candidate record i£ IN(, , . ) of the 
suggested point is false and the candidate point is in the digitized 
video area, and (2) generate a candidate record for the suggested 
point only if both IN and OUT are false and the point is inside the 
digitized video frame. Only option (1) is used in the present sys- 
tem. If a new candidate record is generated, it is put at the end 
of the candidate point records list. 

L7: Continue to Ll. 

D. THE DECISION PROCEDURES 

There are two points where decisions are made and options for decision 
procedure are available: (1) the point where decisions are made to add can- 

didate points to the region, (2) the point where decisions are made whether to 
generate a candidate record for a point adjacent to a point just added to the 
region (see L6 above). The first type of options affects the data structure of 
a candidate record, and it affects steps 14, 15, L3, and L6 of the driver 
algorithm of the blooming region grower. 

1 . Option 1 

Three different decision algorithms were implemented to decide 
whether to add a point to the region. The first and the simplest one requires 
that the region record will includp the running value of the maximum (MAXG) 
and the minimum (MING) of G(i, j) over points already inside the region. If 
(i, j) is the candidate point, it will be added to the region if both predicate 

G(i, j) < MING + TR and G(i, j) > MAXG - TR 

are satisfied, where TR is a decision threshold specified to the algorithm 
before it is called. This decision guarantees that always TR > MAXG - MING 
for the region. The candidate record for that option contains only the i and j 
coordinate of the candidate point. When a point is added to the region, MAXG 
and MING are updated, if necessary, in step L5. 
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2. Option 2 - 

The second decision option is more complicated. It requires mainte- 
nance of a current estimate of the statistics of G(i, j) values in the region 
near the point in the candidate record. The candidate record in this case 
contains the following information: (1) i, j coordinate of the candidate point, 
(2) running average G of G(i, j), and (3) running average G 2 of G(i, j) 2 
(used to compute local variance). 

In step 15, the initial statistics, e.g. , the av^age of G(i, j) and 
G(i, j) around (1^, Jq), are estimated. We are doing that by setting N equal 
to die set of all points (i, j) such that 

N = { (i, j) I ! ‘ - I 0 |< S. I j - J Q | <s, j G(i, j) - G(I 0 , J 0 )| < r} 

where S (the window size) and R (the allowed gray vaLue ran ge) are two 

— 2 

external p> % imeters that control the algorithm. Gq and Gq are then set to: 

E G(i - j) 

G" - (i. J) «N 

# ' E 1 

(i, j) «N 

_ E G(i - j)2 

cl - SLfi 11 

E * 

(i, j) «N 

We probably could have done about as well by initializing. 

S = G <V J o> 

G o = G <V J o> 2 + N 
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where N is the typical variance of the noise of the cameras, which is about 1 
in our case. In any case, the candidate point records list after step 15 will be: 

(i 0 ♦ 1. J #> cj||, G*) 

fa - >• J o- ~o- °o) 
fa» J o + >. G o- G o) 

fa. J 0 - 1. Gq, cl) 

assuming all four points are within the image frame. If in step L6 a new 
candidat e re cord is generated, it will be done as follows. Assume 
^i, j, G, was the candidate record of the point just added to the region, 
and (k, 1) is the point adjacent to (i, j), from which a new candidate record 
is generated. The new candidate record will be (k, l, G', G”) where: 

G' - GU. J> + W » G 
1 + W 

G „ . G(i, j) 2 + W • G Z 
1 + W 

W is the decay factor that is another parameter that controls the algorithm's 

performance. G' and G" are a stochastic estimate of the average G(i, j) and 
2 

G(i, j) of points belonging to the region near the new candidate point. This 
computation requires minimal overhead, especially where 1 + W = 2 n n > 1, 
in which case a simple shift operation will replace the division operation. 

The information in a candidate record (i Q , j Q , G, G 2 ) is used in step L3 
to decide whether to add the candidate point to the region. This is a sequen- 
tial decision process. It is controlled by two arrays of decision thresholds 
and 2 ... n such that always and Hn = Ln where n is 

the maximum number of sample points for the sampler (to be defined later). 
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3. Algorithm D (The Decision Algorithm) 

We assume the candidate point record is (ig, jg, G, G 2 ). 

Dl: VAR = G 2 - G 2 . 

D2: D q = (G(i 0 . j 0 ) - g) 2 . 

D = D 0 . 

D3 : D £ Hg X VAR; then immediately the decision is made not to 

include {ig, jg) in the region. Here G(ig, jg) is too far from the 
average gray level of points of the region near {ig, jg), and the 
decision is made immediately. Note that the threshold Hg, is s 
scaled by the local variance; hence, in some sense, the noise 
and texture of the region are compensated for in the decision. 
This is true for all decisions. 

D4 ; If D £ Lg X VAR then the candidate point is immediately added to 
the regions. 

D5: Initialize N and S; set k = 0. 

Set S = {(ig, jg)} , N = {(ig ■ If jg)» (*q * i» Jq)» ^0' ^0 ” ^0' ^0 ^ 

assuming all four points are in the image range. 

Step D5 is reached where Lq < Dq/VAR < Hg, and a sequential 
decision process has to be initiated. The set S of image points, 
which was already seen by the sampler while working on (ig, jg), 
is initialized to include one point: (ig, jg). From that stage on, 
samples will be taken from the set N which will contain aU points 
adjacent to points in S and not in S (candidate points for the 
sampler). These samples are used to decide whether to add 
(ig, jg) to the region. N is initialized to include four points: 

(ig “ if jg)f (ip + i* jg)» (igf jg i)f atl< i Ugf jg “ !)• 

D6: k = k + 1, 

D7*. Select a sample from N to be the k-th sample. This sample will 
be selected to be the point in N for which G is maximized over 
points of N if G(ig, jg) > G, and to be the point on which G is 
minimized otherwise. 
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in the following steps, we will assume that (i k , 
k-th sample point. , 

DB: D k = (G(i k , - S) 2 . 

D = D + D k . 

D9 : If D £ * VAR, then return immediately with a decision that the 

point (ig, j Q ) is out of the region; otherwise, if D s * VAR, 
return immediately with a decision that (i^, jg) is in the region; 
otherwise, continue to DIO (note the scaling of the decision by the 
local variance). 


sms selected to be the 


DIO ? Add (i k , j k ) to S. Test (i k - l, j fe ), (i fc + 1, j), (i^ j k - 1), and 

(ik, j k + 1); if any of those points is not in S and inside the picture 
frame, add it to N. Go to D7. 


This algorithm is guaranteed to terminate after n iterations since 

L = H 
n n 

The sampling mechanism tries to find the best evidence that (i^, j^) is 
the start of new region of gray level distributions that is uniformly brighter 
or darker than the current one. Most of the decisions are reached in the first 
very few sample points so there is not that much overhead for the decision. 
The sampling is independent of the shape of the new region so that the 
sampling can follow a line of new gray values in an arbitrary direction. 


4. Option 3 

The third option is a modification of the Algorithm D described for 

— "T 

option 2. In addition to the statistics G and G , option 3 requires maintenance 
of statistics involving the i, j coordinates of points in the region near the 
candidate point. The candidate record becomes 

(i, j, G, G 2 , T, J, I 2 , J 2 . 13, Gl, Gj) 

— — 2 2 
where 1 and J are the running averages of i and j, respectively. 1 and J are 

2 2 — 

the running averages of i and j , respectively; IJ is the running average of 
i • j; and C?I and GJ are the running averages of G(i, j)i and G(i, j)j, respectively. 
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Initial statistics Gj| and Gq are obtained in step 15 as described under 
option 2. 

Step 15 is expanded to initialize tiie additional statistics as follows: 


fjj = i 1 ! * iL 


1 


(i, ) ) «N 


= 


E * 

(i» J) «N 

E * 

(i, j) «N 
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(i. i) «w 
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ij = lit j) tN 
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a, THn 
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0(i, j) • j 


gj = 1%^ ) *rc 




1 


where the set N is defined as before. 

The candidate point record list is initialized to be: 

(*0 + lf ^0* G 0' G 0* I 0* J 0’ I 0* J 0* IJ 0' GI o* GJ o) 
(*0 “ j 0’ G 0* ‘ GJ o) 

(v h + ^0' •*** g*o) 

(*0* ^0 ” * ' G 0’ **’» GJ 0 ) 


assuming these four points are within the picture frame. 

These statistic s ar e updated as new candidate records are generated in 
step L6. If (i, j, G, G^, I, J, ~T? t "j^, IJ, GI, GJ^ is the candidate record of 
the point (i, j) just added to the region, and the point (k, 1) is a new candidate 
adjacent to (i, j), then the new candidate record is * 


(k, 1, G«, G”, I', J*, I", 


J", IJ 1 , GI', GJ') 
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where 


I» 


J» 


I" 


J” 


IJ* 


GI* 


GJ' 


i 4- W • I 
1 + W 


. 1 


1 + 


W • J 

w~ 


i 2 + w : j 2 
1 +*w 



✓ 

+ W • J* 

i + w 


ij 4- W • IJ 
1 + W 


_ G(i, j)J_fWjGI 
l + W 


G(i. i) j 4- W - GJ 
= 1 + W 


with G', G", and W as described under option 2. 

When the decision pro cedu re is called in step L3, the statistics in the 

/ — — J ~Z 2 — — I \ 

candidate record u Q , G, G , I, J, I , J , IJ, GI, GJIare used to obtain 

A, B, and C, which define the function 

E(i, j) = Ai 4- Bj 4- C 

E(i, j) is a linear prediction of the grey level value of point (i, j), if the 
point belongs to the current region. E(i, j) is used in place of G in steps D2 
and D8, The coefficients A, B, and C are the values that minimize 


M 


= £ (c(i, j) - Ai - Bj - c)‘ 
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where the turn is taken over points of the region near the candidate point. 

A, B, and C are obtained by solving the system of linear regression equations; 

Al 2 + BU + Cl » GI 

Alj + BJ 2 + CJ » GJ 

aI + B J + C = G 

Mathematically, we are treating the image points as ordered triples 
(i, j, G(i, j)j and finding the best plane fit through the points already belong- 
ing to the region in the neighborhood of (ig, j„). The decision to add (ig, jg) 
to the region is based on the distance of ^ig, j Q , G(ig, jg)^ from this plane, 
scaled by the quality of the plane fit to points already in the region (VAR). 

This allows the algorithm to compensate both for changes in gray level 
intensity due to texture or shading of the object as in option 2, and for 
structured slopes in the grey levels. 

As in option 2, the decision process is controlled by two arrays of 
decision thresholds, | and |h. i = 1, • • • , n, such that a Lu for all i 
and s where n is the maximum number of sample points. 


5. Algorithm E 


EQ: compute A, B ( and C. 

El: VAR a (gUq, jg) - AT - b7 - 

£2: D 0 * ( C <V j 0 > ■ E <V Jo>) 2 - 

G(ig, jg). 



E(ig, jg) is the predicted value of 



E3 : D > Hg * VAR, then immediately the decision is made not to 

include (i Q , jg) in the region. Here, G(ig, j Q ) is too far from the 
predicted gray level based on points of the region near (ig, jg), and 
the decision is made immediately. Note that the threshold Hg is 
scaled by the local variance; hence, in some sense, the noise 
and texture of the region are compensated for in the decision. 
This is true for all decisions. 
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E4 : If D < Lq X VAR, then the candidate point it immediately added 

to the regions. 

E5 *. Initialize N and S; set k * 0, 

Step E5 is reached where Lq < y^ < and a sequential decision 
process has to be initiated. The set S of image points that were 
already seen by the sampler for point (1 q, Jq) is initialized to 
include one picture point (i^, j q). Prom that point on, samples 
will be taken from the set N, which will contain all points adjacent 
to points in S outside of S (candidate points for the sampler). This 
sampling will determine whether to add (i^, j^) to the region, N 
is initialized to include the four points, (i Q - 1, j^), (i Q 4 l, j Q ), 

(!(,* j 0 + l) and ^0* ^0 * 

E6 ; k = k + 1, 

E7; Select a sample point from N to be die k-th sample. This sample 
will be the point (i, j) in N for which 

(c(i, )) - £<i. j)) 2 


is a maximum over the points of N; e.g. , (i^, 
that is furthest from the predicted value, 

£* D k • ( G <V - E <V V)*- 

E9 : If D 2 • VAR, then return immediately with a decision that the 

point (i^, Jq) is out of the region; otherwise, if D 5 * VAR 

return immediately with a decision that (i^, j^) is in the region; 
otherwise, continue to E10 (not the scaling of decision by the 
local variance). 


j k ) is the point N 


E10: Add (i k , j fe ) to S. Test (i fc - 1, j k ), (i fc + 1, j), (i k , - 1), and 

(ifc, j k + 1); if any of those points is not in S and inside the picture 
frame, add it to N. Go to E7. 
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IV. THE USE OF SEQUENTIAL SAMPLING FOR EDGE 
DETECTION FOR THE SCANNING REGION GROWER 


The scanning region grower (object detector), which is reported in 
Ref. 8 (see Subsection IV-D), is based on edge confidence evaluation. This 
edge evaluation is done by an edge detector operator that satisfies two 
requirements. The first is to return a value 0 at most none edge points; the 
second is to return a positive value reflecting the confidence in the existence 
of the edge when it Is applied to a point that is likely to be an edge of an 
object. 

The following technique was developed to upgrade performance of the 
region grower (object locator) based on edge detection, which was reported 
in Ref. 8. The main motive behind the development of the new edge opera* 
tion was performance upgrading along the following lines: 

(1) Efficient detection of points that are obviously not edge points. 

(2) Achievement of independence of the edge value from shape and 
structure of fire objects (lines, sharp corners, etc.). 

(3) Keep the edge evaluation self-scaling with respect to linear 
changes in signal and noise. 

To achieve those seemingly conflicting goals, we used the sequential 
decision approach. 

A. SAMPLE POINT SELECTION 

The edge detector is built around a sampling mechanism (miniregion 
grower) that tries to collect the best evidence for the existence of two differ- 
ent gray level distribution structures around the test point. The sampler 
tries to find a "bright" neighborhood and a "dark" neighborhood around the 
analyzed point. Let S^ be the dark and S Q the bright neighborhood; then, 
on initialization; Sg - S D = P Q where Pq is the analyzed point. Let 
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N b (N d ) be all point* adjacent to S fi (S D ) and not in S B (S D ). The 
•ampler takes the brightest (or darkest) point in N^CN^), puts it in s g(S D ), 
removes it from Ng(Njj), and updates Ng(Np), 

This is an iterative process and our goal is to stop it early if there is 
clearly no edge around Pq and return to value 0, and, if there is an edge, 
return the "strength" of that edge, 

B. STOPPING CRITERIA 

Let (Xj, X^, * * * , X n ) be toe ordered sequence of value readings at the 
points selected in the bright neighborhood, and (Y j, Y^, • ••, V R ) toe sequence 
of readies at the points selected for the dark neighborhood. Our first goal is 
to stop the sampling process if there is no edge. For that purpose we define 
a series of threshold (Tj, T 2 , • • • , T r ) such that if on the i-th iteration 

(X. - Y.) < T. 1 < i < n 

The sampling is stopped and value of 0 (ac-edge) is returned by toe operator. 

To minimize dependence on the properties of the specific image, the 
thresholds are picked based on toe statistics of toe specific image. The 
histogram of g{x, y) - g(x + dx, y + dy) for jdx| + |dy| * 1 is generated by 
sampling these values from the whole image. Then the T^ - s are defined as 
percentage thresholds of that histogr.am. In the liberal case toe T^'s were 
set to toe lowest value such that at least a - 0/i of the pairs sampled has a 
difference in value less than T.. The range of i used was between 1 and 5, 
setting ff *0,9 and 0 » 0. 3 (the percentage threshold starting from 0. 6 to 
0.84) for the liberal case, and lower (by Z) T. values for the conservative 
case. 

If the sampling went all the way up to N (a predetermined integer 
constant set to 5 in our examples) with the differences satisfying the reject 
threshold, then we compute the edge confidence value as follows: 
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and the edge confidence value was defined as 


confidence » 


V 1 V 2 


This value was truncated to 0 if it was less than a certain fixed threshold. 
Thresholds of at least 16 have a good property; if the samples are taken 
from a uniform slope area of gray values, e.g,, 


X, = A + i • B 

i 

and 

Y t = A - i • B 

The edge confidence value is 16. We do not want to define points in a uniform 
gray value slope area as edge points. Results of region growing based on that 
value are shown below. 

Since the confidence value mentioned above is very expensive to com- 
pute. we compare its performance with another edge strength evolution, 

which wag 
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N 

strength * ^ (Xj - Yj) 
i«i 

Results were eubstantialiy better when the variance seeled technique was used, 

C. REMARK ON THRESHOLD SETTING 

Since many cameras are nonlinear in respect to their light sensitivity, 
scaling of the threshold with respect to the intensity of the point may be 
desired. An alternative approach will be to use photon counting cameras 
instead of regular TV tubes for the input device (this instrument measures 
the time it takes to receive a specific number of photons from a given 
direction). 

D. THE REGION GROWER 

The edge operator is applied to each point in the picture and returns a 
value of the confidence that the point is an edge point. This data is used to 
grow regions that are defined as connected valleys of edge values. That is 
let E(P) be the edge value of the point P; then, for a region R, Pq is a 
minimum edge value point. For example, 

E(P.) . min jE(P)| 

P«R 

and Q is another point of R; then there is a path P^, Pj (P ft * Q) of points 
from R such that P i is adjacent to P. + j ami E(P.) s E(P i+i ). A one-pass 
algorithm that grows those regions is described in detailr in Ref, 8. 
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V. EXAMPLES 


This section shows examples of the region grower options described in 
this paper applied to three scenes (Figs. 4 through 24); these scenes were 
taken from the RAMTEK RX-100A unit. Examples of a scanning region 
grower are also included. Each scene has been segmented four times using 
the options described in this paper as follows: 


(1) 

Option 1: 

TR = 12. 

(2) 

Option 2: 

^0 = H 0 = 13 

(3) 

Option 2: 

L 0 =4, H 0 < 16 

(4) 

Option 3; 

L q =4. H 0 *16 


n = 20, L 2q = H 20 - 4, W = 5. 
n = 20, L 20 =H 20 = 10, W * 15. 
n = 20, L 2q = H 2q • 10. W . 5. 


Notice that option 2 was used twice, first with "conservative" thresholds, 
and then with "liberal" thresholds. With the conservative thresholds, there 
is a tendency to stop before reaching the actual region boundaries, but points 
that do not belong to the region are rarely included. With the liberal 
thresholds, there is a tendency to include points that do not belong to the 
region, but points that do belong to the region are rarely excluded. For 
example, notice how the sky has been segmented in Figs. 6 and 7. We feel 
that by applying a 'Combination of region grower options and/or thresholds to 
a scene, by a semantic system, segmentation errors can be detected and a 
good set of region boundaries can be obtamed for the scene. 


Each scene also has been segmented twice with a sequential decision 
based scanning region (Section IV) grower, using conservative and liberal 
thresholds, to show a comparison of the methods. 
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4. Outdoor scene 


Fig. 5. Segmentation with option 1 


Fig. 6. Segmentation with 
option 2 — conservative 


Fig. 7. Segmentation with 
option 2 — liberal 
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Fig. 10. Segmentation with scanning 
region grower — conservative 


Fig. 8. Segmentation with option 3 


Fig. 9. Segmentation with scanning 
region grower — liberal 
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Fig. 13. Segmentation with Fig. 14. Segmentation with 

option 2 — conservative option 2 — liberal 


A face 


Fig. 12. Segmentation with option 1 
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Fig. 15. Segmentation with option 3 


Fig. 16. Scanning region grower 
segmentation — liberal 
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Fig. 17. Scanning region grower 
segmentation — conservative 
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Fig. 20. Segmentation with Fig. 21. Segmentation with 

option 2 — conservative option 2 — liberal 
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